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(57) ABSTRACT 

An object identification method for wireless portable 
devices for a user equipped with a portable wireless imaging 
device to be able to obtain information and services related 
to imaged objects, where the object identification is per- 
formed at least partially by a remote computational facility, 
and where the object identification is based on acquired 
images of the object. The method includes an imaging 
device, capable of taking one-dimensional or two dimen- 
sional images of objects; a device capable of sending the 
coded image through a wireless channel to remote facilities; 
algorithms and software for processing and analyzing the 
images and for extracting from them symbolic information 
such as digits, letters, text, symbols or icons; algorithms and 
software facilitating the identification of the imaged objects 
based on the information gathered from the image and the 
information available in databases; and algorithms and soft- 
ware for offering various information or services to the user 
of the imaging device based on the information gathered 
from the image and the information available in databases. 
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FIG.2 is a view of the processing flow for a 
sample application of the invention: 




A barcoded object is scanned 
through e.g. a Visor PDA 
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FIG.3 is a view of the processing flow for 
another sample application of the invention: 

—— . . ~„ §bf2frtuf}ork Stows ~ 



A newspaper object is 
photographed 



The image is enhanced 





The headline is extracted 



The relevant page is displayed 
on the browser 
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FIG.4 is a description of the data flow in the 
system during operation 



An object is 
photographed using 
1D or 2D camera 




The camera sends the data to 
the wireless section using a 
cable or an infrared port or a 
wireless connection 
(bluetooth). 




The wireless device sends 
compressed images through th 



The images are stitched 
together 



The text is extracted & sent 
The image is rotated to the to ® e ™ ral com P^rs on to 
right reading angle Kodak comhe server for 
• M parallel processing 




009900363532 
Orbit sugarfree The extracted text 
gum is sent to other 

computers for 

lingual 
interpretation 
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OBJECT IDENTIFICATION METHOD FOR 
PORTABLE DEVICES 

[0001] The present application is based on Serial No. 
60 245,661 filed on Nov. 6, 2000, the contents of which are 
hereby incorporated by reference. 

BACKGROUND OF THE INVENTION 
[0002] 1. Field of the Invention 

[0003] The present invention relates generally to wireless 
imaging technology and more specifically it relates to an 
object identification method for wireless portable devices for 
a user equipped with a portable wireless imaging device to 
be able to obtain information and services related to imaged 
objects, where the object identification is performed at least 
partially by a remote computational facility, and where the 
object identification is based on an acquired image or images 
of the said object. The imaging device and the wireless 
device can be one entity, as in a cellular phone or PDA with 
an integrated camera, or they can be two separate devices, as 
in a digital camera connected by wire or wirelessly to a 
cellular phone or other wireless transmission device. 

[0004] 2. Description of the Related Art 

[0005] It can be appreciated that object or printed material 
identification technology has been in use for years. Typi- 
cally, an object identification system is comprised of: 

[0006] 1. An OCR, watermark or barcode analysis 
software running on a PC, Workstation or a dedi- 
cated processing engine. 

[0007] 2. Portable devices equipped with an imaging 
device and computational engine for performing 
OCR, barcode or watermark analysis on the scanned/ 
acquired images. The main problem with conven- 
tional devices for image based object identification is 
that for high quality identification powerful software 
is required. This software utilizes high-end proces- 
sors and large quantities of memory, and hence 
typically runs on a non-portable device such as a 
personal computer. As an example for such a system 
is the watermark identification system called Medi- 
aBridge™ developed by Digimarc Inc, where the 
processing is done on a PC. 

[0008] In a special purpose portable and/or wireless 
device, the computation capacity is much lower, and hence 
the recognition task is simplified by using higher quality, 
special purpose image acquisition and/or by decoding sim- 
pler codes. Examples for these tradeoffs and their solutions 
include: 

[0009] 1. Standard barcodes, sampled by a bar-code 
reader, featuring a dedicated illuminator and/or 
detector optimized for the task of linear bar-code 
decoding. Pertinent examples include the barcode 
readers made by ConnectThings, DigitalConver- 
gence, Gamut-interactive etc. These devices cannot 
decode anything by a standard barcode. 

[0010] 2. For performing reliable OCR using limited 
performance software, one may incorporate into the 
system a high quality, special purpose linear scanner 
such as the one used in the Quicktionary™ product 



by WizCom. With a special purpose scanner the 
OCR task becomes simpler. 

[0011] 3. One can limit the OCR functionality to a 
very limited set of alpha-numeric characters in a 
limited set of fonts. Hence the processing and 
memory requirements are reduced, making the 
implementation portable. The Quicktionary™ and 
Cpen™ devices are examples. 

[0012] Another problem with conventional devices for 
image based object identification is that the portable devices 
perform lower grade recognition (such as OCR) because of 
power, size and price constraints, and hence give the user a 
limited capacity in terms of handling difficult imaging 
conditions, low grade print or handwriting, special fonts and 
different languages. Portable devices are also harder to 
upgrade when new versions of software become available. 

[0013] Another problem with conventional devices for 
image based object identification is that the portable devices 
are special purpose and hence have to be purchased and 
carried separately to provide only this function. Further- 
more, many of these devices are not connected on-line to the 
Internet or other on-line data bases, and licncc cannot 
provide real-time or semi-real-time connection to data based 
on the scanned image, text or symbols. 

[0014] It should be mentioned that devices for sending 
wireless images are now becoming commonplace. Some 

[0015] 1. The Nokia 9110 Cellular phone is capable 
of interfacing using an IrDA port to a digital camera 
and sending the image. 

[0016] 2. Lightsurf Inc. has a system for a special 
purpose camera attached to a cellular phone. 

[0017] 3. ActivePhoto Inc. is making devices and 
software for attaching numerous digital cameras to 
cellular phones. 

[0018] 1. Cpen is making a device for scanning 
text/images and sending it to a cellular phone by the 
BlueTooth™ wireless protocol. 

[0019] 5. Ericsson is working with Cannon to make 
a cellular phone and camera system. 

[0020] While these devices may be suitable for the par- 
ticular purpose to which they address, they are not as 
suitable for a user equipped with a portable wireless imaging 
device to be able to obtain information and services related 
to imaged objects, where the object identification is per- 
formed at least partially by a remote computational facility, 
and where the object identification is based on acquired 
images of the object. 

[0021] In these respects, the object identification method 
for wireless portable devices according to the present inven- 
tion substantially departs from the conventional concepts 
and designs of the prior art, and in so doing provides an 
apparatus primarily developed for the purpose of a user 
equipped with a portable wireless imaging device to be able 
to obtain information and services related to imaged objects, 
where the object identification is performed at least partially 
by a remote computational facility, and where the object 
identification is based on acquired images of the object. 
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[0022] Some relevant patents that represent the prior art 

[0023] 1. In the field of algorithms and image pro- 
cessing operations for removing the effects of imag- 
ing under uncontrolled illumination and with low 
quality/limited imaging devices, there are numerous 
patents, see e.g. U.S. Pat. No. 5,771,312, incorpo- 
rated herein by reference. The implementation of 
such existing algorithms and methods in the remote 
server for improving the image quality for human 
observers are also not new. The concept of develop- 
ing and optimizing such algorithms as part of a 
remote server for improving the accuracy of the 
object identification is new. 

[0024] 2. Many algorithms exist for performing 
printed and hand-typed character recognition based 
on images, see e.g. U.S. Pat. Nos. 5,359,671, 6,011, 
879, 4,977,602, 5,542,006, each of these four patents 
is incorporated herein by reference. In the method 
according to the invention, one inventive aspect lies 
in utilizing such algorithms for performing object 
identification rather than e.g. performing word iden- 
tification as part of inputting a printed page into a 
computer as text. 

[0025] 3. There is also significant prior art on using 
special marks or codes such as barcodes, watermarks 
etc for object identification, see e.g. U.S. Pat. Nos. 
5,978,733, 5,933,829, each of these two U.S. patents 
is incorporated by reference. On the other hand, the 
inventive method uses standard marks such as 
numerals or text that appeared on the object for 
human reading, and emulating the human method of 
object identification. The limitation of using special 
marks is that access to the full world of objects that 
were not marked specifically for automated identi- 
fication is not available. For example, in the case of 
scanning barcodes, the inventive method does not 
require a dedicated barcode scanner but rather it uses 
a standard imaging device, and it interprets the data 
contained in the barcode based on both the lines and 
the digits rather than based solely on the lines. 

SUMMARY OF THE INVENTION 
[0026] In view of the foregoing disadvantages inherent in 
the known types of object identification technology now 
present in the prior art, the present invention provides a new 
object identification method for wireless portable devices 
construction wherein the same can be utilized for a user 
equipped with a portable wireless imaging device to be able 
to obtain information and services related to imaged objects, 
where the object identification is performed at least partially 
by a remote computational facility, and where the object 
identification is based on acquired images of the object, and 
potentially on other additional information, such as (but not 
limited to) the device/user location, user profile, previous 
user actions, and the user's textual, manual or acoustic 

[0027] The general purpose of the present invention, 
which will be described subsequently in greater detail, is to 
provide a new object identification method for wireless 
portable devices that has many of the advantages of the 
ob ject identification technologies heretofore and many novel 



features that result in a new object identification method for 
wireless portable devices which is not anticipated, rendered 
obvious, suggested, or even implied by any of the prior art 
of wireless imaging technology, either alone or in any 
combination thereof. 

[0028] To attain this, the present invention generally com- 
prises: 

[0029] 1. An imaging device, capable of taking one- 
dimensional or two-dimensional images of objects. 

[0030] 2. A device capable of sending the coded image 
through a wireless channel to remote facilities. 

[0031] 3. Algorithms and software for processing and 
analyzing the images and for extracting from them 
symbolic information such as digits, letters, text, sym- 
bols or icons 

[0032] 4. Algorithms and software facilitating the iden- 
tification of the imaged objects based on the informa- 
tion gathered from the image and the information 
available in databases. 

[0033] 5. Algorithms and software for offering various 
information or services to the user of the imaging 
device based on the information gathered from the 
image and the information available in databases. 

[0034] The imaging device is a unit capable of acquiring 
images, storing and/or sending them. The wireless device is 
capable of sending images to remote facilities. The algo- 
rithms perform compression artifact correction, noise reduc- 
tion, color corrections, geometric corrections, imager non- 
uniformity correction, etc., and various image processing 
enhancement operations to better facilitate the operation of 
the next stage of image understanding algorithms. The 
algorithms are implemented as a plurality of software 
objects residing on one or more computational devices. 
Algorithms performing, among other operations, digit rec- 
ognition, printed and handwritten text recognition, symbol, 
logo and watermark recognition, and general texture and 
shape recognition. The algorithms are implemented as a 
plurality of software objects residing on one or more com- 
putational devices. Also included is software for utilizing the 
information extracted in the previous computation stages for 
data storage, extraction and/or communication with a plu- 
rality of internal and/or external applications, such as data- 
bases, search engines, price comparison sites etc. Also 
included is software for sending relevant information and/or 
services back to the user by any means. 

[0035] The invention may include, in certain embodi- 
ments, algorithms for determining where and by which 
computational device the processing will be carried, based 
on parameters such as device loads, capabilities, network 
conditions, security constraints, etc. 

[0036] The invention may include, in certain embodi- 
ments, algorithms for determining that the automatic object 
recognition has failed or that the results are suspect, or that 
the user desires human intervention, or that the user has 
specified that he wants human recognition to be applied, and 
direct the visual or other information gathered to a system 
where human beings may perform the recognition task or 
utilize partially automatic algorithms to accomplish the 
same goal. 
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[0037] The invention may also include, in certain embodi- 
ments, software for assisting, instructing and informing the 
user through, for example a graphical user interface, of the 
various stages of operation such as proper image capture, 
alignment, wireless link availability etc. 

[0038] There has thus been outlined the more important 
features of the invention in order that the detailed descrip- 
tion thereof may be better understood, and in order that the 
present contribution to the art may be better appreciated. 
There are additional features of the invention that will be 
described hereinafter. 

[0039] In this respect, before explaining at least one 
embodiment of the invention in detail, it is to be understood 
that the invention is not limited in its application to the 
details of construction and to the arrangements of the 
components set forth in the following description or illus- 
trated in the drawings. The invention is capable of other 
embodiments and of being practiced and carried out in 
various ways. Also, it is to be understood that the phrase- 
ology and terminology employed herein are for the purpose 
of the description and should not be regarded as limiting. 

[0040] A primary object of the present invention is to 
provide an object identification method for wireless portable 
devices that will overcome the shortcomings of the known 
devices. 

[0041] An object of the present invention is to provide an 
object identification method for wireless portable devices for 
a user equipped with a portable wireless imaging device to 
be able to obtain information and services related to imaged 
objects, where the object identification is performed at least 
partially by a remote computational facility, and where the 
object identification is based on acquired images of the said 
object. 

[0042] Another object is to provide an object identification 
method for wireless portable devices that utilizes a cellular 
phone, personal digital assistant or other device equipped 
with an imaging device and with connectivity to other 
computational sources such as the internet, to provide 
advanced image recognition and understanding services 
using remote computational facilities for performing OCR, 
barcode and logo analysis. 

[0043] Another object is to provide an object identification 
method for wireless portable devices that performs image 
pre-processing to correct for image artifacts created by the 
imaging conditions which apply to imaging a substantially 
planar surface (such as a sheet of printer paper, a product 
label, a sticker etc.) in various uncontrolled illumination 
conditions such as those found in normal day to day envi- 
ronments, where the imaging device is a camera or linear 

[0044] Another object is to provide an object identification 
method for wireless portable devices that performs image 
pre-processing to correct for image artifacts generated by the 
imaging optics, electronics, compression and/or communi- 
cation error correction schemes for one of the above men- 
tioned devices. 

[0045] Another object is to provide an object identification 
method for wireless portable devices that performs image 
enhancement using multiple still images or image sequences 



or video sequences to improve the image quality for one of 
the above mentioned devices. 

[0046] Another object is to provide an object identification 
method for wireless portable devices that utilizes computa- 
tional models involving a remote computational facility 
("server") and distributed processing in this facility to 
provide faster response times. 

[0047] Another object is to provide an object identification 
method for wireless portable devices that utilizes the infor- 
mation extracted from the image to detect the imaged object, 
and using this information connects the user with informa- 
tion, web sites or telephone numbers related to this object. 

[0048] Another object is to provide an object identification 
method for wireless portable devices that utilizes the infor- 
mation extracted from the image to store, send or manipulate 
a description on this object in a non-image format, e.g. a text 
string, a digit string, or a code. 

[0049] Other objects and advantages of the present inven- 
tion will become obvious to the reader and it is intended that 
these objects and advantages are within the scope of the 
present invention. 

[0050] To the accomplishment of the above and related 
objects, this invention may be embodied in the form illus- 
trated in the accompanying drawings, attention being called 
to the fact, however, that the drawings are illustrative only, 
and that changes may be made in the specific construction 
illustrated. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0051] FIG. 1 is an exploded view showing the various 
components of an embodiment of the invention; 

[0052] FIG. 2 is a processing flow chart according to an 
embodiment of the invention; 

[0053] FIG. 3 is a processing flow chart according to an 
embodiment of the invention; 

[0054] FIG. 4 is a data flow chart according to an embodi- 
ment of the invention. 

DETAILED DESCRIPTION OF THE 
PREFERRED BODIMENT 

[0055] Various other objects, features and attendant 
advantages of the present invention will become fully appre- 
ciated as the same becomes better understood when consid- 
ered in conjunction with the accompanying drawings, in 
which like reference characters designate the same or similar 
parts throughout the several views. 

[0056] FIG. 1 is an exploded view of an embodiment of 
the present invention showing all the system components. 

[0057] Item 101 is the imaging device, as described pre- 
viously. In terms of novel additions, the imaging device may 
contain image compression algorithms specially optimized 
for the task of image compression for optimal identification 
rather than optimal appearance. For example, for the iden- 
tification of printed text/numerals. The system can convert 
the image into a binary black and white image for better 
compression, even though it makes the image less visually 
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appealing. Potentially, the device may run specific soft- 
ware — e.g. code written In J2ME, to optimize the image 
taking operation. 

[0058] In item 102 the imaging operation is performed 
through the camera Field Of View (FOV). Part of the novelty 
of the invention lies in the understanding thai through the 
remote server one can stitch several images to form the 
complete image required for identifying the object (see also 
FIG. 4). 

[0059] Item 103 represents a potential identifying mark, 
such as a barcode. One aspect of the novel method is that the 
barcode is not read using a specially designed device but 
rather using a general purpose imaging device. 

[0060] Item 104 represents another potential identifying 
mark, such as the printed text in a document. In the case of 
e.g. a newspaper, the headings or even just fragments of text 
in a story/advertisement could serve as identifying informa- 

[0061] In item 105, once the image or set of images is 
acquired it is transmitted through any wireless/wireline 
combination oi'dala transmission paths to the remote server. 
The remote server could be far apart e.g. in the central office 
of a wireless cellular operator, or it could be a few meters 
away from the imaging device and connected to it by a 
WLAN such as Bluetooth. 

[0062] Item 106 is the remote server, which then proceeds 
to apply the described sequence of algorithms, which can be 
a combination of known and novel algorithms. Appendix A 
provides a detailed description of the algorithms for barcode 
detection and decoding. The processing server applies such 
sequences of algorithms that result in the identification of 
the imaged object. 

[0063] Item 107 is the remote server itself (or a different 
remote server connected to it). Server 107 can, based on the 
object identification information, extract information about 
the object from databases/public data networks such as the 
internet. For example, the ISBN number of a book could be 
used to perform an HTTP GET request to a web site such as 
Amazon in order to retrieve the product's price, reviews 

[0064] FIG. 2 is a view of the processing flow for a sample 
application of the invention. 

[0065] Item 201 is the imaging device (as described as 
item 101 in FIG. 1). 

[0066] Item 202 is the image of a standard UPC barcode 
on a commercial product. 

[0067] Item 203 is the part of the image that has been 
extracted by either the imaging device or by the remote 
server and contains the information necessary for object 
identification. The algorithms required to implement this 
stage are described in Appendix A. 

[0068] Item 204 is the string of identifying numbers that 
has been extracted using algorithms such as those described 
in Appendix A. 

[0069] Item 205 is the server, which then formulates e.g. 
an HTTP request or a database SQL query to retrieve more 
information about the product — e.g. price, availability, 
qualities, rating, limitations on sale etc. 



[0070] Item 206 is the target device. The retrieved infor- 
mation is then reformatted for display on the screen of the 
target device — so for example graphics may be taken out or 
reduced in color depth or size before they are sent to the 
device 206, and the binary format in which they are pack- 
aged has to be adapted to the recipient device. This can be 
done by the remote server or by a different entity. 
[0071] In item 207 the server's response may include 
menu options and perform activities on the display device, 
so the product can be bought or inquired about. 

[0072] Item 208 shows that the same information (or more 
information) can also be sent aft to other display devices 
such as the user's personal computer, e-mail account etc. 
This can enable richer interaction at a later time when the 
user is near a more powerful device. The content is adapted 
in any case to the different target devices. 

[0073] FIG. 3 is a view of the processing flow for another 
sample application of the invention. 

[0074] Item 301 is the imaging device (as described as 
item 101 in FIG. 1). 

[0075] Item 302 is the image of a part of a newspaper 
page. 

[0076] Item 303 is the image after image processing 
operations have been performed on it to decrease the file size 
and/or improve the object identification's chances. In this 
example the image is binarized after some local histogram 
equalization operations. 

[0077] In item 304, the OCR engine running on the remote 
server identifies the part of the image containing legible text 
and extracts the maximum number of characters and their 
relative geometrical position. This information is then used, 
in conjunction with a database of the newspaper itself, to 
identify the relevant story /segment. It should be noted that 
for identification purposes even a very partial success in the 
character recognition task should be sufficient. In item 305, 
again the results are reformatted and transcoded optimally to 
the target device — which is not necessarily the original 
imaging device 301. 

[0078] FIG. 4 is a description of the data flow in the 
system according to one embodiment of the invention. 

[0079] Item 401 is the imaging device (as described as 
item 101 in FIG. 1). 

[0080] Item 402 is another potential imaging device with 
a line scanner rather than a two dimensional imager. 

[0081] Item 403 is the data transmission apparatus in cases 
where the image acquisition part of the device is connected 
to the data transmission apparatus through a cable or some 
special wireless connection. 

[0082] Item 404 is the original acquired image prior to any 
manipulation. 

[0083] Item 405 is the compressed image prior to sending, 
where the image compression parameters and algorithm may 
have been optimized for object identification purposes rather 
than for visual appeal. 

[0084] Item 406 is the remote server system, which may 
be comprised of a series of servers where the image pro- 
cessing operations between these servers are distributed 
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(either on a per image basis or on a per-request basis) for 
optimizing the computational resources and/or the total 
response time. The distribution may be performed via com- 
mercial load balancing equipment or by proprietary load 
balancing software. 

[0085] Items 407 and 408 are two separate images that 
have been acquired and can be stitched together in the 
remote server to form one complete image. 

[0086] Item 409 the image is then rotated to the right angle 
for OCR detection (see Appendix A for a detailed discussion 
of this operation), where the algorithm measures the image 
angle using the line pattern of the barcode. 

[0087] In item 410, the part of the image containing 
numerals is extracted using a special algorithm (see Appen- 
dix A). 

[0088] In item 411, OCR operations then take place on the 
remote server, where again parallel processing may take 
place to enable testing many more image parameter con- 
figurations or OCR fonts, or several different OCR engines 
may be run in parallel and the final result determined by 
some form of voting mechanism. 

[0089] In item 412, the string of the decoded numbers 
(which may contain some errors) is sent for interpretation to 
better decide e.g. the type of the barcode (UPC,EAN, some 
proprietary formal tic). Some error correction algorithms 
may be used at this stage to utilize the inherent redundancy 
in the digits to correct for identification errors. Finally, in 
item 413, the extracted text is sent to other computer lingual 
interpretation. 

[0090] Based on the above figures, the object identifica- 
tion method for wireless portable devices includes the 
following operations: 

[0091] 1. An imaging device, capable of taking one- 
dimensional or two-dimensional images of objects. 

[0092] 2. A device capable of sending the coded 
image through a wireless channel to remote facili- 

[0093] 3. Algorithms and software for processing and 
analyzing the images and for extracting from them 
symbolic information such as digits, letters, text, 
symbols or icons. 

[0094] 4. Algorithms and software facilitating the 
identification of the imaged objects based on the 
information gathered from the image and the infor- 
mation available in databases. 

[0095] 5. Algorithms and software for offering vari- 
ous information or services to the user of the imaging 
device based on the information gathered from the 
image and the information available in databases. 

[0096] The imaging device 101 is a unit capable of acquir- 
ing images, storing and/or sending them. The wireless 
device is capable of sending images to remote facilities. The 
algorithms perform compression artifact correction, noise 
reduction, color corrections, geometric corrections, imager 
non-uniformity correction, etc., and various image process- 
ing enhancement operations to better facilitate the operation 
of the next stage of image understanding algorithms. The 
algorithms are implemented as a plurality of software 



objects residing on one or more computational devices. Also 
included are algorithms performing, among other opera- 
tions, digit recognition, printed and handwritten text recog- 
nition, symbol, logo and watermark recognition, and general 
shape recognition. The algorithms are implemented as a 
plurality of software objects residing on one or more com- 
putational devices, possibly including the imaging device 
and/or the wireless device. Software for utilizing the infor- 
mation extracted in the previous computation stages for data 
storage, extraction and/or communication with a plurality of 
internal and/or external applications, such as databases, 
search engines, price comparison sites etc. 
[0097] The imaging device 101 is a unit capable of acquir- 
ing images, storing and/or sending them. The imaging 
device is a device capable of capturing single or multiple 
images or video streams and converting them to digital 
information. It is equipped with the proper optical and 
electro-optical imaging components and with computational 
and data storage components. The imaging device can be a 
digital camera, a PDA with an internal or external camera, 
a cellular phone with an internal or external camera, or a 
portable computational device (e.g. laptop, palmtop or Web- 
pad™-like device) with an internal or external camera. 

[0098] The wireless device is capable of sending images to 
remote facilities. The wireless device is a device capable of 
transferring information wirelessly to remote or nearby 
locations. It is capable of getting the information from the 
imaging device for processing and transmission. It can also 
be capable of receiving information wirelessly or using a 
wired connection. It can also be capable of performing some 
processing operations reducing the load of sending the raw 
image to the remote server or even of reducing the compu- 
tational load on the server by performing other image 
processing and image analysis operations. The wireless 
device can be a cellular phone, a wireless PDA, a Web- 
pad™-like device communicating on a local wireless area 
network, a device communicating using infrared or acoustic 
energy, etc. 

[0099] The algorithms perform compression artifact cor- 
rection, noise reduction, color corrections, geometric cor- 
rections, imager non-uniformity correction, etc., and various 
image processing enhancement operations to better facilitate 
tile operation of tile next stage of image understanding 
algorithms. The algorithms are implemented as a plurality of 
software objects residing on one or more computational 
devices. The image processing algorithms are numerical and 
symbolic algorithms for the manipulation of images and 
video streams. The algorithms perform compression artifact 
correction, noise reduction, color corrections, geometric 
corrections, imager non-uniformity correction, etc., and 
various image processing enhancement operations to better 
facilitate the operation of the next stage of image under- 
standing algorithms. The algorithms are implemented as a 
plurality of software objects residing on one or more com- 
putational devices. The algorithms can be implemented as 
software running on a general purpose processor, DSP 
processor, special purpose ASIC and/or FPGA's. They can 
be a mixture of custom developed algorithms and libraries 
provided by other developers or companies. They can be 
arranged in any logical sequence, with potential changes in 
the sequence of processing or parameters governing the 
processing determined by image type, computational 
requirements or outputs from other algorithms. 
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[0100] Another aspect of the invention is a collection of 
algorithms performing, among other operations, digit rec- 
ognition, printed and handwritten text recognition, symbol, 
logo and watermark recognition, and general shape recog- 
nition. The algorithms are implemented as a plurality of 
software objects residing on one or more computational 
devices. The image processing algorithms are numerical and 
symbolic algorithms for the manipulation of images and 
video streams. The algorithms perform, among other opera- 
tions, digit recognition, printed and handwritten text recog- 
nition, symbol, logo and watermark recognition, and general 
shape recognition. The algorithms are implemented as a 
plurality of software objects residing on one or more com- 
putational devices. The algorithms can be implemented as 
software running on a general purpose processor, DSP 
processor, special purpose ASIC and/or FPGA's. They can 
be a mixture of custom developed algorithms and libraries 
provided by other developers or companies. They can be 
arranged in any logical sequence, with potential changes in 
the sequence of processing or parameters governing the 
processing determined by image type, computational 
requirements or outputs from other algorithms. The algo- 
rithms may reside on a different system belonging to a 
different entity than the image processing algorithms or the 

[0101] Another aspect of the invention is software for 
utilizing the information extracted in the previous compu- 
tation stages for data storage, extraction and/or communi- 
cation with a plurality of internal and/or external applica- 
tions, such as databases, search engines, price comparison 
sites etc. The application software provides the overall 
functionality of the service, based on the information 
extracted in the previous algorithmic stages. It is software 
for data storage, extraction and/or communication with a 
plurality of internal and/or external applications, such as 
databases, search engines, price comparison sites etc. The 
application software can be implemented as code running on 
a general purpose processor, DSP processor, special purpose 
ASIC and/or FPGA's. It can be a mixture of custom devel- 
oped software and libraries provided by other developers or 
companies. This software may reside on a different system 
belonging to a different entity than the rest of the system. 

[0102] The imaging device captures one or more images 
or video sequences, which are (potentially) processed on this 
device and then transferred to the wireless device or the 
wireless transmission section of the complete device. The 
data is then transmitted and transferred through some kind of 
data network or networks to servers which process the 
information using the above-described algorithms, and then 
uses the extracted information for various applications. The 
servers (or other connected entities) may then send infor- 
mation back through the network to the wireless device, or 
to other devices such as a personal computer or set-top box. 
A large portion of the processing algorithms may be reside 
on the portable device, and there may be a dynamically 
changing division of the algorithms running on the different 
parts of the system based on relative computational loads 
and desired user response times, changing imaging and 
wireless bandwidth conditions. The application software 
executing for a given image or image sequence may be 
determined based on the image content itself, rather than 
being fixed. The application software to be used may be 
chosen by the user based on pre-configured parameters or 



during the operation. The outputs of the application software 
may be sent back to the user through any channel. 

[0103] The principle of operation is that using images or 
image sequences or video sequences, a computer can decode 
the identity of the imaged object, for example a labeled 
product, a printed form, a page from a book or newspaper, 
a bill, a membership card, a receipt, a business card, a 
medical prescription etc. This saves the user the time and 
effort of inputting the object identity and/or unique infor- 
mation pertaining to the object such as values in numerical 
fields, addresses in a business card, etc. The imaging device 
captures images or video sequences, which are (potentially) 
processed on this device and then transferred to the wireless 
device or the wireless transmission section of the complete 
device. The data is then transmitted and transferred through 
a data network or networks to servers which process the 
information using the above-described algorithms, and then 
uses the extracted information for various applications. The 
servers (or other connected entities) may then send infor- 
mation back through the network to the wireless device, or 
to other devices such as a personal computer or set-top box. 
[0104] With respect to the above description then, it is to 
be realized that the optimum dimensional relationships for 
the parts of the invention, to include variations in size, 
materials, shape, form, function and manner of operation, 
assembly and use, are deemed readily apparent and obvious 
to one skilled in the art, and all equivalent relationships to 
those illustrated in the drawings and described in the speci- 
fication are intended to be encompassed by the present 
invention. 

[0105] Therefore, the foregoing is considered as illustra- 
tive only of the principles of the invention. Further, since 
numerous modifications and changes will readily occur to 
those skilled in the art, it is not desired to limit the invention 
to the exact construction and operation shown and 
described, and accordingly, all suitable modifications and 
equivalents may be resorted to, falling within the scope of 
the invention. 

Appendix A: Algorithms for Barcode Detection and 
Extraction 

[0106] This is a description of the algorithms relevant for 
utilization of an image of a barcode on the object in order to 
identify the object by its barcode number. 

[0107] The algorithm consists of 6 main steps (that will be 
described in details in the following paragraph): 

[0108] 1) Identify the barcode in the image, by recog- 
nizing regions in the image which resemble barcodes 
(uniformity in one axis and change in the other, etc.) 
regardless of the image rotation, the tilt of the image 
plane to the camera and the scale (to a reasonable 
extent). 

[0109] 2) Based on the above identification, recognize 
the dimensions, orientation and location of the barcode. 

[0110] 3) Extract a normalized image strip of the digits 
accompanying the barcode — this strip is now of con- 
stant size and is not skewed. 

[0111] 4) Read the digits in the extracted strip, achiev- 
ing improved quality by utilizing the barcode specific 
information: relative location of digits, fonts, barcode 
checksum. 
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[0112] 5) Combining the OCR results with a direct 
optical reading of the barcode's lines, using super- 
resolution, will further enhance accuracy of reading. 



[0113] 6) Invoking an application specific operation, 
based on the identified product id (e.g. presenting the 
web page for this product). 
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Library barcode 



[Keywords | Classes | Data | Functions ] 

Quick Index 

CV EXTRACT BARCODE 
APPLIES BARCODE FILTER 
BARCODE BEST CANDIDATE 
BARCODE ANGLE 

BARCODE WIDTH APPROXIMATION 
ROTATE AND TRANSLATE 
BARCODE HEIGHT 
BARCODE . DIGITS 
TILT PARAMETERS 





Classes 


Back to the top. of barcode 






Data 


Back to the top. of barcode 





Global Functions 



bool get barcode angle flpllmage* src_img_8u,lpllmage* dst_img_8u,lpllmage* 
ratio_img_8u,IplImage* tpl_img_8u,lpllmage* match Jmg_32f, int xc,int 
yc,double& rad_angle); 

bool act tx^st barcode candidate posdpllmage* src_img_8 ujpl Image* 
dst_img_8u, int& xc,int& yc,int& area); 

void get barcode width aprox flpllmage* src_img_8ujpllmage* dst_img_8u,mt 

xc,int yc,double angle,int& x_left,int& x_right); 

bool get barcode h eighttlpllmage* src_img_8u,lpllmage* 

dst_img_8u,lpllmage* temp_img_8u,int& barcode_height); 

bool ap ply barcode filter on ro idpllmage* src_img_8u, Ip 1 Im age* 

dstjmg_8u,lpllmage* tpl_img_8u,lpllmage* match_img_32f,lpllmage* 

ratio_img_8u,IplROI* region_roi,IplROI* tpl roi iplROI* match_roi, double& 

eig_val_min,double& eig_val_max,double& eig_angle); 

void app Iv barcode fi Iter flpllmage* src_img_8 u, Ip 11m age* 

dst_img_8u,lpllmage* ratio_img_8u,IplImage* tpl_img_8u,lpllmage* 

match__img_32f); 

hit cv correctji Ih lpllmage* src_img_8u,lpllmage* dst_img_8u,double 
xc,double ycl,double yc2,doub!e angle); 
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bool find digit bottom line flpllmage* src_img,lpllmage* dst_img,int 
barcode_height,int& digit_height,float& deg_tilt_angle); 

v °id enhance white uniform linesffpllmage* src_img_8u , Ip 1 Image * 

dst_img_8u,int high_pass_width); 

bool compute tilt parametcrs QplImage* src_img_8u,Tpl Image* 

dst img_8u,lpllmage* temp_img_8u,int barcode_height,int& digit_height,float& 

deg_tilt_angle); 

int £ct digit height f Ipllmage* src Jmgjpllmage* dst_img,lpllmage* 
temp_img,float* line_param); 

int get digit bottom line flpllmage* srcimgjpllmage* dst img, float* 
line_param); 

int extract bareode (IplImage* src_mig_8u,Ip]Image* dst_img_8u,lpllmage* 
ratio_img_8u,IplImage* tpl_img_8u,lpllmage* match_img_32f, int& xjeft,int& 
x_right,int& y_up,int& y_down); 

void get max contour ellipse flpllmage* src_img,CvBox2D32f& ellipse); 
void contrast stretch flpllmage* src_img,lpllmage* dst_img); 
void histeq flpllmage* src_img,lpllmage* dst_img); 

void translate barcode flpllmage* src_img_8u,lpllmage* dst img_8u,int xc,int 

yc); 

void rotate bareode frpllmage* src_img_8u,lpllmage* dst_img_8u,int xc,int 
yc,double angle); 

Back to the top. of barcode 



bool get_barcode_angle(IplImage* src_img_8u,lpllmage* 
dst_img_8u,lpllmage* ratio img 8u,lpllmage* tpl_img_8u,lpllmage* 
match img 32f, int xc,int yc,double& rad_angle); 
#iD elude " cv barcode an2le.h " 

computes barcode angle xc and yc are the barcode centroid coordinates returns the 
barcode angle in radians 

b ° o1 get barcodejingle (Ipl Image* src_img_8u, Ipl Image* dst_img_8u, Ipllmage* 
ratio_img_8u, Ipl Image* tpl_img_8u, Ipllmage* match_img_32f , 

int xc,int yc, doubles 

rad_angle) ; 

Back to the top of barcode 



bool get best barcode_candidate_pos(IplImage* src_img_8u,lpllmage* 
dst_img_8u, int& xc,int& yc,int& area); 
#include " cv barcode candidate.h " 

computes the biggest connected component of the barcode filter output image returns the 
best candidate centroid coordinates and its area 

b ° o1 .get best barcode candidate pos (Tpl Tmagp* src_img_8u, Ipllmage* dst_img_8u, 
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int& xc,int& yc,int& area); 
Back to the tog of barcode 



void getbarcode width aprox(IplImage* src img 8u,lpllmage* 
dst_img_8u,int xc,int yc,double angle,mt& x_Ieft,int& x_right); 
#include " cv barcode dim.h " 

computes barcode margins returns the barcode left and right approximated margins 

void get barcode \vidth_aprox (Ipl Image* src_img_8u, Ipl Image* dst_img_8u, int 
xc,int yc, double angle, int& x_left,int& x_right) ; 

Back to the tog of barcode 



bool get_barcode_height(IplImage* src_img_8u,lpllmage* 
dst_img_8u,lpllmage* temp inig_8u,int& barcodeheight); 
#include " cv barcode dim.h " 

computes barcode height 

bool get barcode height ( Ipl Image* src_img_8u, Ipl Image* dst_img_8u, Ipllmage* 
temp_img_8u, int& barcode_heigh.t) ; 

Back to the top of barcode 



bool appIy_barcode_filter_on_roi(IplImage* src img 8u,lpllmage* 
dst img 8u,lpllmage* tpl img 8u,lpllmage* match_img_32f,lpllmage* 
ratio_img_8u,IpIROI* region_roi,IpIROI* tpl_roi,IplROI* match_roi, 
doub!e& eig_val_min,double& eig_val_max,double& eig angle); 
#include " cv barcode filter.h " 

applies barcode filter on region of interest returns the eigen values of the correlation line 
and the angle of the principal axes 

bool apply barcode filter on roi (Ipllmage* src_img_8u, Ipllmage* 
dst_img_8u, Ipllmage* tpl_img_8u, Ipllmage* match_img_32f , Ipllmage* 
ratio_img_8u, IplROI* region_roi , IplROI* tpl_roi , IplROI* matchjroi, 

doubles 

eig_val_min, doubles eig_val_max, doubles eig_angle) ; 



Back to the top of barcode 
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void apply_barcode_filter(IplImage* src_img_8u,lpllmage* 

dst_img_8u,lpllmage* ratio_img_8u,IplImage* tpl_img_8u,lpllmage* 

match_img_32f); 

#include " cv barcode filter.h " 

applies barcode filter on overalapping blocks 

void apply barcode filter (Ipl Image* src_img_8u, Ipllmage* dst_img_8u, Ipllmage* 
ratio_img_8u, Ipllmage* tpl_img_8u, Ipllmage* match_img_3 2f ) ; 

Back to the top of barcode 



int cv_correct_tilt(IplImage* src_img_8u,lpllmage* dst img 8u,double 
xc,double ycl,double yc2,double angle); 
#include " cv correct tilth " 

applies an affme transform to correct the tilt deformation xc is the barcode centroid x 
coordinate ycl is y coordinate of the end of the barcode lines yc2 is the y coordinate of 
the digit bottom line 

int cv correct tilt ( Ipl Image * src_img_8u, Ipllmage* dst_img_8u, double 
xc, double ycl, double yc2, double angle); 

Back to the top of barcode 



bool find digit_bottom line(lpllmage* src_img,lpllmage* dst_img,int 
barcode_height,int& digit_height,float& deg tilt angle); 
#include " cv correct tilth " 

finds barcode digit bottom line return the digit bottom line angle and its y coordinate 

bool find digit bottomjine (Ipllmage* src_img, Ipllmage* dst_img, int 
barcode_height, int& digit_height , floats deg_tilt_angle) ; 

Back to the top of barcode 



void enhance_white_uniform_lines(IplImage* src_img_8u,lpllmage* 
dst_img_8u,int high_pass_width); 
#include " cv correct tilth " 

applies wide high pass filter to enhance the long white lines 

void enhance white uniform lines (Ipllmage* src_img_8u, Ipllmage* dst_img_8u, int 
high_pass_width) ; 



Back to the top of barcode 
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bool compute_tilt_parameters(IplImage* src_img_8u,lpllmage* 
dst img 8u,IpIImage* temp_img_8u,int barcode height,int& 
digit height,float& deg tilt angle); 
#in elude " cv correct tilth " 

computes tilt parameter from the digit bottom line returns the digits height and the angle 
of the digit bottom line 

bool compute tilt parameters (Ipl Image* src_img_8u, Ipllmage* 
dst_img_8u, Ipllmage* temp_img_8u, int barcode_height, int& 
digitjieight, floats deg_tilt_angle) ; 

Back to the top. of barcode 



int get_digit_height(lpllmage* src_img,lpllmage* dst_img,lpllmage* 
temp_img,float* line_param); 
#include " cv digit height.h " 

all images are uint8 240x320; srcjmg is an gray image that contains the area bellow the 
upper bound of the barcode digits dst_img contains the barcode digits templ_img is a 
temporary image; returns the barcode digit height 

int get digit height (Ipllmage* src_img, Ipllmage* dst_img, Ipllmage* 
temp_img, float* 1 ine_param) ; 

Back to the top of barcode 



int get_digit_bottom_line(IpIImage* src_img,lpllmage* dst_img,float* 

lineparam); 

#include " cv digit heighth " 

all images are uintS 240x320; src_img is a binary image that contains the area bellow the 
upper bound of the barcode digits dst_img contains the barcode digits templ_img is a 
temporary image; returns the lower bound of the barcode digits 

int get digit bottom line t Ip 1 Image * src_img, Ipllmage* dst_img, float* 
line_param) ; 

Back to the top of barcode 



int extract_barcode(IpIImage* src_img_8u,lpllmage* 
dst_img_8u,lpllmage* ratio_img_8u,IpIImage* tpl_img_8u,lpllmage* 
match img_32f, int& x_left,int& x_right,int& y_up,int& y down); 
#include " cv extract barcode.h !t 
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extracts barcode xjeft and x_right are barcode horizontal borders y_up is the end of the 
barcode lines y_down is the end of the barcode digits barcode is rotate until is horizontal 

int extract barcode (Ipl Image* src_img_8u, Ipllmage* dst_img_8u, Ipl Image* 
ratio_img_8u, Ipllmage* tpl_img_8u, Ipllmage* match_img_32f , 

int& x_left,int& x_right, int& 

y_up, intt y__down) ; 
Back to the tog of barcode 



void get_max_contour_ellipse(IplImage* src_img,CvBox2D32f& ellipse); 
#include " cv misc flinch " 

all images are uint8 240x320; 

void get max contour ellipse (Ipllmage* src_img, CvBox2D32f & ellipse); 
Back to the top. of barcode 



void contrast_stretch(lplImage* src imgjpllmage* dst img); 
#include " cv misc flinch " 

all images are uint8 240x320; stretches the contrast of an image 
void contrast stretch (Ipl Image* src_img, Ipllmage* dst_img) ; 

Back to the top of barcode 



void histeq(lpllmage* src_img,lpllmage* dst img); 
#include " cv misc funch " 

all images are uint8 240x320; perform an histogram equalization 
void histeq (Ipllmage* src_img, Ipllmage* dst_img) ; 

Back to the top of barcode 



void translate_barcode(IplImage* src_img_8u,lpllmage* dst_img_8u,int 
xc,int yc); 

#include " cv traas rot barcode.h " 

translates barcode to the upper border of the image 

void translate barcode (Ipllmage* src_img_8u, Ipllmage* dst_img_8u, int xc,int 
yc) ; 



Back to the top of barcode 
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void rotate barcodeflpllmage* src_img_8u,lpllmage* dst_img_8u,int 
xc,int yc,double angle); 
#include " cv trans rot barcode.h " 

rotates barcode until is horizontal 

void rotate barcode (Ipl Image* src_img_8u, Ipl Image* dst_img_8u, int xc,int 
yc, double angle) ,- 

Back to the top of barcode 
CVEXTRACTBARCODE 

the library contains functions to extract the barcode from a given input image 
there are several steps to perform in order to extract the barcode digits 
Back to the top of barcode 

APPLIES BARCODE FILTER 

- applies barcode filter - looks for the areas in the image where a barcode is present 
the input image 



9lill|l|! 



the filter size is 32x32. it runs on ovelapping blocks and perform NCC with a template 
8x8 extracted from the center of each block. 




the cross correlation map 
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after closing the internal holes with some morphological operations the cross correlation 
map is binarized with an absolute threshold 

Back to the top of barcode 
BARCODEBESTCANDIDATE 

- finds barcode best candidates - looks for the connected components of the barcode 
candidates ;their centroids and areas are computed; the biggest one is chosen 




output image 

Back to the top of barcode 



BARCODEANGLE 

- computes the barcode angle - computes the barcode orientation with a maximal error of 
1 degree 

performs NCC with a template 32x32 around the best barcode candidate centroid 
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From the NCC image result the parameters of the biggest line are computed. 
Back to the top of barcode 

BARCODE_WIDTH_APPROXIMATION 

- computes the barcode width approximation 




input image is the binary image of the best barcode candidate 




the barcode candidate is rotated until becomes horizontal output image 
Back to the top of barcode 
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ROTATEANDTRANSLATE 

- rotates and translate the barcode - the barcode is rotated until becomes horizontal; after 
that its centroid is translated to the point (img_width/2,0) 




input image for rotation 




rotated barcode 



translated barcode 

Back to the top_ of barcode 



BARCODEHEIGHT 

- computes the barcode height - detects the end of the barcode lines 
performs NCC with a template 1x64 extracted from the point (width/2,10) 

J1IS111 
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input image 



NCC result 




output image 

Back to the top. of barcode 



BARCODE_DIGITS 

- computes the barcode digits height - detects the lower bound of the barcode digits 




input image 

the image contrast is increased by homogenization and after that is binarized. 




the white long lines below the point where the barcode lines end 
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the long white lines are enhanced by a wide high pass filter 

a vertical scan line is dropped from the barcode height point until the digit bottom line is 
crossed. 




output image, the digit line is marked by an empty circle and the centerline of the digits is 
drawn. 

Back to the top of barcode 
TILTPARAMETERS 

- computes tilt parameters from the digit bottom line - apply an affine transform to 
correct the tilt deformation 




Back to the top of barcode 
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1. A system for acquiring basic information about a 
particular object of interest, for transmitting and receiving 
said basic electronic information, for identifying the object 
from said basic electronic information, for transmitting and 
receiving additional information or services, and for dis- 
playing said additional information, said system comprising: 

(a) an imaging device for acquiring said basic information 
about the object; 

(b) a communication device for transmitting the basic 
information to a remote server and receiving additional 
information about the object; 

(c) a remote server for receiving said basic information 
about the object, lor processing said basic information 
to identify the particular object of interest, to acquire 
additional information about the object of interest, and 
for transmitting said additional information to said 
communication device; 

(d) application software that allows the remote server to 
identify the object of interest; and 

(e) application software that allows the remote server to 
acquire additional information about said object. 

2. The system set forth in claim 1 further comprising a 
wireline communication link between the communication 
device and the remote server. 

3. The system set forth in claim 1, wherein the additional 
information is services. 

4. The system set forth in claim 1 wherein the imaging 
device is separate from the communication device in ele- 
ment b, but these two devices are linked electronically. 

5. The system set forth in claim 1 wherein the imaging 
device and the communication device are comprised of only 
one device that performs both imaging and communication. 

6. The system set forth in claim 2 wherein the commu- 
nication device is a PDA. 

7. The system set forth in claim 3 wherein the commu- 
nication device is a PDA. 

8. The system set forth in claim 4 wherein the imaging 
device is a PDA. 

9. The system set forth in claim 5 wherein the single 
device for imaging and communication is a PDA. 

10. The system set forth in claim 3 wherein the commu- 
nication device that performs the communication link with 
the remote server is a cellular telephone. 

11. The system set forth in claim 4 wherein the commu- 
nication device is a cellular telephone. 

12. The system set forth in claim 5 wherein the single 
device for imaging and communication is a cellular tele- 
phone. 

13. The system set forth in claim 1 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

14. The system set forth in claim 2 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

15. The system set forth in claim 3 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 



profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

16. The system set forth in claim 4 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

17. The system set forth in claim 5 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

18. The system set forth in claim 6 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

19. The system set forth in claim 7 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

20. The system set forth in claim 8 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

21. The system set forth in claim 9 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

22. The system set forth in claim 10 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

23. The system set forth in claim 11 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

24. The system set forth in claim 12 wherein the basic 
information includes at least one piece of information from 
the group consisting of the device/user location, the user 
profile, previous user actions, the user's textual inputs, the 
user's manual inputs, and the user's acoustic inputs. 

25. At least one portable device comprising: 

(a) a means for acquiring at least one image that includes 
at least one object of interest; 

(b) a means for transmitting to a remote computational 
facility data that includes data associated with said at 
least object of interest; 

(c) a means for receiving processed data from said remote 
facility to enable (possibly with the application of 
additional calculations by the portable device) identi- 
fication of said at least one object; 

(d) a means for receiving additional information or ser- 
vices about said at least one object; and 

(e) a means for displaying said processed data and addi- 
tional information or services about said at least one 
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26. A method for identifying at least one object and 
providing additional information or services about at least 
said one object, comprising: 

(a) acquiring at least one image that includes at least one 
object of interest; 

(b) transmitting to a remote computational facility data, 
that includes data associated with said at least object of 



(c) receiving processed data from said remote facility to 
enable identification of said at least one object; 

(d) receiving additional information or services about said 
at least one object; and 

(e) displaying said processed data and additional infor- 
mation or services about said at least one object. 



